Automated plant trimmer

ABSTRACT

The automated trimming of an untrimmed plant stem comprises a device for transporting an untrimmed plant stem to an imaging stage, using a camera to capture at least one image of the plant stem, sending the at least one image to a central processor which creates a trim map which constructs a pattern for moving a cutting armature along the plant stem, and trimming the plant stem by cutting structure of the cutting armature to change the untrimmed plant stem to a trimmed plant stem.

FIELD OF INVENTION

The present invention relates generally to the field of automatedpruning and trimming of plants and crops such as (but not limited to)Cannabis sativa and Cannabis indica (“cannabis”). More specifically, butwithout limitation, embodiments of the present invention includemachines, apparatuses, systems and methods for automated or roboticscanning, mapping, picking, pruning, trimming, manipulating anddetecting diseases or parasites associated with agricultural crops,including without limitation cannabis.

BACKGROUND

After the growing phase is complete, cannabis plants are harvested bytrimming off branches that include a long, central stem, large “fan”leaves (also called water leaves), and a cola (also called a bud orflower), in which numerous smaller leaves (called sugar leaves) are alsointerspersed. The next step in the harvest process is the removal of fanleaves and sugar leaves, either before or during the drying process, sothat what is left are the parts of the plant—the buds or flowers,including trichomes (see below)—containing a higher concentration ofactive compounds, primarily tetrahydrocannabinol (THC), cannabidiol(CBD) and numerous other cannabinoids. These active compounds areparticularly concentrated in the flower and in the nearly microscopic,translucent resin glands on the flower surface called trichomes. Thetrichomes and the flower are delicate structures and easily damaged orbroken through handling. Thus, one objective of the trimming process isto remove as many of the low-cannabinoid leaves as possible withoutdamaging or removing the high-cannabinoid flower and trichomes.

The trimming process has been traditionally accomplished using manuallabor, in which an individual uses a pair of small scissors to cut awaythe fan leaves, sugar leaves and other low-cannabinoid parts of theplant. Manual trimming, however, is slow, monotonous, imprecise,expensive and exposes valuable (and otherwise tightly controlled)inventories of cannabis flowers to theft by workers and damage in thetrimming process.

In response to these drawbacks (including but not limited to damage totrichomes, tedious manual labor, damage to the product, reduction inproduct margins and shrinkage through theft), various mechanical deviceshave been created to trim harvested plants, including the TRIMPRO RotorXL (www.trimpro.com), the CENTURION PRO Silver Bullet(www.cprosolutions.com), the TWISTER T4 (www.twistertrimmer.com), theULTRATRIMMER (www.ultratrimmer.com), the GreenBroz (www.greenbroz.com)and the THUNDERVAK Composter Plus. All are similar in operation—arotating, reciprocating or stationary chamber containing the harvestedcannabis stalks includes small slits or gaps in which—theoretically, butnot in practice—only fan and sugar leaves (but not other parts of theplant) can slip easily, and behind each slit or gap is a mechanicalcutting edge. The stalks are rotated or agitated inside the chamber, andwith each cycle, the stalks come into contact with the slits, protrudingleaves are sheared off, and the process repeats until complete. Inpractice, mechanical trimming with these machines results inover-trimming of the harvested stalk and significant damage to theflower and trichomes, which renders the most valuable parts of the plantuseless and converted into waste.

For other agricultural applications, computer-controlled crop pickers,harvesters and trimmers have been created—for example, U.S. Pat. No.9,226,446, to Moore (“Robotic Fruit Tree Pruner and Harvester Machine”)and U.S. Pat. Pub. No. 2011/0022231 to Walker et al. (“Apparatuses,Systems and Methods for Automated Crop Picking”). Both referencesdisclose machine-vision systems coupled with computer-controlledmechanical armatures for pruning and harvesting fruit and othertree-borne crops. These devices, however, lack numerous featuresrequired or advantageous for the trimming of cannabis stalks, asdescribed in greater detail below.

Specifically, the use of computer-vision for trimming plants and otheragricultural products presents a unique challenge. In a typicalcomputer-vision application, the central processor is programmed withinstructions to search for and compare certain shapes, sizes and colorsagainst a library of reference images, which permit the machine to“recognize” certain parts or features and act upon them. When workingwith plants and crops, however, reference images are far less usefulbecause of wide variations in the natural shape, color and location ofleaves, stalks, flowers, buds and other plant parts. Therefore, adifferent approach to computer-vision is beneficial.

SUMMARY

According to the present invention, there is provided an automatedtrimming device including a transport mechanism, a cutting supportmechanism, a camera and imaging stage and a cutting armature or cuttinglaser in communication with a central processor.

The transport mechanism includes a series of cars mounted to amechanically driven conveyor system for moving untrimmed plant stemsfrom a supply source into an appropriate position adjacent to thecutting support mechanism for trimming. The cutting support mechanismincludes a platform for supporting the untrimmed plant stem while thestem is imaged and trimmed. The camera and imaging stage are positionedto capture at least one image of the plant stem in its untrimmed stateand, from that at least one image, the central processor computes anappropriate map for trimming the stem. The central processor then usesthe trim map to guide the cutting armature into various positions fortrimming the plant stem.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front perspective view of an exemplary embodiment of anautomated plant trimmer.

FIG. 2 is a side perspective view of an exemplary embodiment of anautomated plant trimmer.

FIG. 3 is a top view of an exemplary embodiment of an automated planttrimmer.

FIG. 4 is a side perspective view of several portions of an exemplaryembodiment of an automated plant trimmer—in particular, exemplaryembodiments of a transport mechanism, cutting support mechanism, camera,imaging stage, cutting armature and central processor.

FIG. 5 is a front perspective view of exemplary embodiments of a cuttingsupport mechanism, imaging stage and cutting armature.

FIG. 6 is a front view of exemplary embodiments of a cutting supportmechanism, imaging stage and cutting armature.

FIG. 7 is a detail view of an exemplary embodiment of a cuttingarmature.

FIG. 8 is a front view of exemplary embodiments of a cutting supportmechanism, imaging stage and cutting armature, showing movement of thecutting armature.

FIG. 9 is an overhead view of an exemplary embodiment of a cuttingsupport mechanism, imaging stage and cutting armature.

FIG. 10 is an overhead view of an exemplary embodiment of a cuttingsupport mechanism, imaging stage and cutting armature, showing movementof the cutting armature and cutting support mechanism.

FIGS. 11A, 11B and 11C are overhead views of an exemplary embodiment ofa cutting support mechanism, imaging stage and cutting armature, showingmovement of the cutting armature and cutting support mechanism.

FIG. 12 is a cutaway view of an exemplary embodiment of a transportmechanism.

FIG. 13 is a detail view of an exemplary embodiment of a transportmechanism, camera, cutting support mechanism and imaging stage.

FIG. 14 is an image of a plant stem prior to trimming, as captured by acamera.

FIG. 15 is a comparison of images of a plant stem in the original stateand in the silhouette or outline state.

FIG. 16 is an image illustrating the shape-fitting process against asilhouette image of a plant stem prior to trimming.

FIG. 17 is an image illustrating the shape-fitting process aftersubtraction of certain portions of the image previously covered bycircles or other shapes.

FIG. 18 is an image illustrating the branch identification processagainst a silhouette image of a plant stem prior to trimming.

FIG. 19 is a flow chart illustrating the steps of an embodiment of imageprocessing.

FIG. 20 is an image illustrating the shape recognition process against asilhouette image of a plant stem prior to trimming.

FIG. 21 is an image illustrating several image processing steps (shapefitting, pattern recognition and leaf tip recognition) against asilhouette image of a plant stem prior to trimming.

FIG. 22 is a side perspective view of a second exemplary embodiment ofan automated plant trimmer.

FIG. 23 is a front perspective view of a second exemplary embodiment ofan automated plant trimmer.

FIG. 24 is a rear perspective view of a second exemplary embodiment ofan automated plant trimmer.

FIG. 25 is a top view of a second exemplary embodiment of an automatedplant trimmer.

FIG. 26 is a bottom view of a second exemplary embodiment of anautomated plant trimmer.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Reference will now be made in detail to various embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction with thefollowing embodiments, it will be understood that the descriptions arenot intended to limit the invention to these embodiments. On thecontrary, the invention is intended to cover alternatives, modificationsand equivalents that may be included within the spirit and scope of theinvention as defined by the appended claims. Furthermore, in thefollowing detailed description, numerous specific details are set forthin order to provide a thorough understanding of the present invention.However, it will be readily apparent to one skilled in the art that thepresent invention may be practiced without these specific details. Inother instances, well-known methods, procedures, components andmaterials have not been described in detail so as not to unnecessarilyobscure aspects of the present invention.

The invention, in its various aspects, will be explained in greaterdetail below with regard to exemplary embodiments. References to thedrawings include reference characters designating like or correspondingparts throughout several views.

As illustrated in FIGS. 1 and 4, the embodiment depicted in the attachedFIGS. includes the following principal structures, each described ingreater detail below: a transport mechanism 100, a cutting supportmechanism 200, a camera 300 and imaging stage 400, a cutting armature500 and a central processor 600. The central processor 600 may be anIntel® Core i7 processor or a Jetson TX2 GPU based computer, for examplerunning Microsoft® Windows or Linux and National Instruments LabVIEWsystem design programming environment or another programming languagesuch as ROS (Robot Operating System), C++, Python, OpenCV or a similarlanguage.

The embodiment depicted in the attached FIGS. 1-4 is a robotic transportand plant trimming system that uses data gathered by the camera 300 andother sensors and instructions provided by a central processor 600 (orsome intermediate device in communication with the central processor) toguide a cutting armature 500. Additional data may also be provided bycapacitive, resistive, touch and pressure sensors, which can detectplant water content, physical resistance to the cutting armature, andother physical qualities, among other things. Additionally, the cuttermay or may not be mechanical in nature, and may include a CO₂ or otherlaser source, guided by either a gantry or an articulated mirror.

As illustrated in FIGS. 2, 3 and 12, the transport mechanism 100includes a series of cars 102 mounted to a mechanically driven conveyorsystem 104. Each car 102 holds one or more plant stems 106 in arotatable chuck or collar 108 for gripping and supporting the plant stemor stems as the car 102 travels along the conveyor system 104 and intothe correct position for plant trimming. The chuck 108 rotates along anaxis that corresponds approximately to the long axis of the plant stemor stems mounted in the chuck. The chuck's rotation is driven through afriction or gear wheel system 110 that engages a drivewheel when the caris positioned in the plant-trimming position. The drivewheel is drivenby a motor 112 controlled by electrical signals or instructions receivedfrom a central processor 600 (or some intermediate device incommunication with the central processor). The conveyor system is alsodriven by a motor 114 controlled by electrical signals or instructionsreceived from the central processor 600 (or some intermediate device incommunication with the central processor). An untrimmed plant 106 stemmay be loaded into each chuck 108 by hand or using an automatic loadingmechanism, which would include jaws or fingers for gripping anindividual plant stem from a hopper or other form of supply system,pushing it into the opening of an available chuck and then releasing theplant stem. Automated loading of a plant stem or stems may also beaccomplished through other methods and mechanisms known in the art,including but not limited to the use of an automated six-axis roboticarmature, in an agitated hopper, using automated branch identificationand picking and similar devices.

As illustrated in FIGS. 5, 6, 8, 9 and 10, the cutting support mechanism200 includes a platform 202, preferably made from a transparent ortranslucent material, or a substrate suitable for laser cutting of thesupported material, for supporting the untrimmed stem 106 (with all ofthe attached leaves and flowers) during the imaging and cutting process.The plant stems 106 are expected to be of differing weights, lengths andstiffness, and so it is difficult to predict the direction and extent agiven plant stem will droop or bend as it is moved into theplant-trimming position. A significantly drooping or bending plant stem,in turn, makes the process of imaging and trimming more difficult, asthe areas to be trimmed are more easily distinguished and trimmed whenthe plant stem is approximately perpendicular to the cutting armature500. The platform 202 is preferably transparent or translucent to enablelight to illuminate the plant stem from below the platform, therebycreating a more accurate image of the plant stem in the system's camera.The platform is also preferably mounted slidably, with a spring-drivenbias mechanism 204 holding the platform in the most extended position.As shown in FIGS. 6, 8, 9, 10, 11A, 11B and 11C, this spring-biasedslide mechanism 204 enables the platform 202 to retract as the cuttingarmature 500 moves closer to the plant stem 106 and pushes the platform202 out of the way. The plant stem 106 does not droop or bend when thecutting armature 500 is engaged in cutting at a close distance, becausethe cutting armature 500 itself provides a small amount of support tothe plant stem 106 as the cutting occurs. Once the cutting process isfinished and the cutting armature 500 retracts, releasing pressure onthe support platform 202, the platform 202 slides back to its mostextended position, again supporting the length of the plant stem in thecorrect orientation. There is also a flat plate or fence 206 positionedperpendicular to the support platform 202 that provides a boundary orstop for ensuring that the plant stem 106 does not get pushed out ofposition during the cutting process. Other supporting platforms may alsobe used to hold the plant stem in the correct orientation, includingplatforms made from non-transparent materials, platforms made fromperforated, hex-pattern materials or other materials conducive to lasercutting, platforms made from light-emitting materials and platforms thatare fixed rather than slidably mounted. Furthermore, there may be nosupporting platform in the device if, for example, the plant stem isoriented vertically and suspended from above, as shown in the embodimentin FIGS. 22 to 26. This embodiment is discussed separately below.

As shown in FIG. 13, the camera 300 and imaging stage 400 arepositioned, for purpose of capturing at least one image, immediatelyabove and below the plant stem 106 and support platform 200 when thestem 106 is in the correct plant-trimming position. Other images fromdifferent angles may also be captured to improve the detail and accuracyof the trimming process. Additionally, the camera 300 can be mounted onthe end of the cutting armature 500 in the form of a 6-axis arm, asshown in FIGS. 22 to 26. This embodiment is discussed separately below.The camera 300 is a typical electronic sensor camera with a digitalimage output, such as a camera with a CCD or CMOS image sensor.Preferably, the camera 300 may be a high resolution (megapixel)monochrome or chromatic machine vision camera and may be augmented by asecond, higher quality camera, a 3-D camera or hyperspectral camera,structured light, laser sensor, or LIDAR sensor for improved detail,image and pixel size. The camera 300 may be mounted to a stationaryarmature 302 and primarily positioned so that the camera is pointed in adirection approximately perpendicular to the axis of the plant stem tobe trimmed. The camera 300 or a secondary camera may be mounted on thenon-stationary cutting armature or a separate moving armature or robotarm to provide additional views of the plant stem 106. Below (or behind,if the plant stem is oriented vertically) the plant stem is the imagingstage 400. The imaging stage 400 includes a backlighted surface 402 forilluminating the plant stem to be imaged and trimmed and severalreference markers 404 to be used by the imaging software for determiningthe exact position of the plant stem 106 and the cutting armature 500 onthe imaging stage 400. A backlighted surface 402 may transmit at aspecific wavelength or color to properly differentiate the plant, forexample a blue or white light. Additionally, the backlighted surface 402may be augmented or replaced by a front light, ultraviolet light,infrared light, LED, quartz halogen, fluorescent, xenon strobe or anyother form of light source. The support platform 202 is positionedbetween the plant stem and the imaging stage, but its position in thislocation does not interfere with the image capture process, as thesupport platform 202 is preferably made from a transparent ortranslucent material. Other arrangements of lighting position, cameraposition and imaging stage such that the plant stem is well lit andvisible to the camera's lens are also possible, including the removal ofthe platform, and the arrangement of the plant stem in a verticalorientation, as in FIGS. 22 to 26, or any other orientation, includingon a moving or stationary conveyor. The lighting system may also useother forms of filtered, colored or specialized lighting, such asinfrared filtering, ultraviolet filtering, laser or coherent lightsources or other forms of light beneficial for image capture.

As illustrated in FIGS. 6, 7 and 8, the cutting armature 500 may includea hollow cylindrical chamber 502 including an axial slit or opening 504and cutting structure 506 which could be a laser cutter or a rotatingblade positioned inside the chamber so that leaves or other plantmaterial that slip into the axial slit 504 are cut by the cuttingstructure 506 and thus severed from the plant stem 106. Preferably, asource of suction or lower air pressure 508 is also applied to theinterior of the cylindrical chamber 502 so that leaves, stems, seeds andother plant parts that come into close proximity with the cylinder 502are sucked into the axial slit 504 and then severed when the laser orrotating blade 506 passes across it. The source of low-pressure air 508may be a typical household or industrial wet/dry vacuum, for example.When the cutting structure 506 is a rotating blade, the blade may be inthe shape of an enlongated helix, for example, and may be constructed soas to permit easy replacement when the blade inevitably gets dull fromuse. The cylindrical chamber may also include a stationary blade 512located adjacent to the axial slit 504 with its cutting edge positionedin close engagement to the cutting edge of the rotating blade 506. Thesuction or lower air pressure source 508 then serves to remove the wastetrimmings from the cylindrical chamber 502, where their continuedpresence might otherwise slow the cutting process. Preferably, the wastetrimmings are conveyed by the air flow from the cylindrical cuttingchamber 502 to a waste cup or filter cup (not included in FIGS.), wherethey can be collected and removed at an opportunity convenient to themachine operator. These waste trimmings may be discarded or saved forfurther processing (for example, cannabis oil extraction) to remove therelatively low concentration of cannabinoids from these parts of theplant. The cutting armature 500 may also comprise a mechanical scissors,shears, blade or any equivalent mechanism for cutting, grinding orremoving plant material. A mechanical scissor, for example, may utilizea single or double acting pneumatic cylinder for actuation.

As illustrated in FIGS. 4-5, the cylindrical chamber 502 of the cuttingarmature 500 is oriented along an axis that is substantiallyperpendicular to the plane of the imaging stage 400 and perpendicular tothe axis of the plant stem 106. One end of the cylindrical chamber 502slides along the surface of the imaging stage 400, while the other endis connected to a mechanism 510 for moving the cutting armature to anyspecified location or locations on the surface of the imaging stage. Themechanism 510 for moving the cutting armature may include motors, belts,gears and a communications circuit for receiving electrical signals orinstructions from the central processor 600 (or some intermediate devicein communication with the central processor 600) and translating thesesignals into movement of the cutting armature 500. The mechanism 510 mayalso include motors and linkages for rotating the axial slit 504 in thecylindrical chamber 502 to another position. This would enable thecentral processor 600 to tilt the slit's opening clockwise orcounterclockwise along the long axis and thereby enable the cuttingarmature 500 to be more accurate in trimming certain areas of the plantstem 106, such as right near the flower. During the imaging process, thecentral processor 600 may give instructions for moving the cuttingarmature 500 away from the plant stem 106 so that the position of thecutting armature does not overlap or otherwise interfere with the imageof the plant stem to be trimmed. The cylindrical chamber 502 of thecutting armature 500 may also include capacitive, resistive, touch orpressure sensitive sensors 514 (FIG. 7) for the purpose of gatheringadditional data about the plant stem or trimmed areas, for example thewater content of the plant stem. Alternatively, the cutter could consistof a whip mechanism, for example a thin piece of wire or plasticrotating at a high speed, just as a garden weed trimmer. Alternatively,the cutter may be a non-contact non-mechanical device, such as a CO₂laser or other laser source.

FIGS. 22-26 illustrate an alternative embodiment of an automated planttrimmer, including the following principal structures, each described ingreater detail below: a transport mechanism 100, a camera 300, animaging stage 400 and a cutting armature 500. As in the embodimentdescribed above, the embodiment depicted in FIGS. 22-26 is a robotictransport and plant trimming system that uses data gathered by thecamera 300 and other sensors and instructions provided by a centralprocessor (or some intermediate device in communication with the centralprocessor) to guide a cutting armature 500. The transport mechanism 100includes a series of cars 102 mounted to a mechanically driven conveyorsystem 104. Each car 102 holds one or more plant stems 106 in arotatable chuck or collar 108 for gripping and supporting the plant stemor stems as the car 102 travels along the conveyor system 104 and intothe correct position for plant trimming. The chuck 108 rotates along anaxis that corresponds approximately to the long axis of the plant stemor stems mounted in the chuck. The chuck's rotation is driven through afriction or gear wheel system 110 that engages a drivewheel when the caris positioned in the plant-trimming position. The drivewheel is drivenby a motor 112 controlled by electrical signals or instructions receivedfrom a central processor (or some intermediate device in communicationwith the central processor). The conveyor system is also driven by amotor 114 controlled by electrical signals or instructions received fromthe central processor (or some intermediate device in communication withthe central processor).

As illustrated in FIGS. 22-26, there is no cutting support mechanism asin the embodiment described above, as the plant stem 106 hangsvertically from the car 102 and the chuck 108, which reduces thepotential for the plant stem to droop or bend out of position.

As shown in FIG. 22, the camera 300 in this embodiment is mounteddirectly to the cutting armature 500, where it can be moved to differentpositions and angles of view, thereby improving the detail and accuracyof the trimming process. Behind the untrimmed plant stem 106 is theimaging stage 400. The imaging stage 400 includes a backlighted surface402 for illuminating the plant stem to be imaged and trimmed, which mayalso transmit at a specific wavelength or color to properlydifferentiate the plant, for example a blue or white light.

Additionally, the backlighted surface 402 may be augmented or replacedby a front light, ultraviolet light, infrared light, LED, quartzhalogen, fluorescent, xenon strobe or any other form of light source.The lighting system may also use other forms of filtered, colored orspecialized lighting, such as infrared filtering, ultraviolet filtering,laser or coherent light sources or other forms of light beneficial forimage capture.

As illustrated in FIGS. 22-26, the cutting armature 500 may includesix-axis robotic armature with a mechanical scissor 520 utilizing asingle or double acting pneumatic cylinder for actuation. The plant stemis positioned so that trimmings from the cutting process will fall intoan opening or chute 522 in the floor of the device for collecting suchmaterial for further processing or disposal. In the proximity of thecutting armature 500, there may also be provided an ultrasonic solventbath 524, where the cutting armature may dip the mechanical scissor 520,blade or other cutting implement for cleaning and removing of plantresidue.

As illustrated in FIGS. 22-26, the cutting armature includes a mechanismfor moving the the mechanical scissor 520 to any specified location orlocations on the untrimmed plant stem. The mechanism for moving thecutting armature may include motors, belts, gears and a communicationscircuit for receiving electrical signals or instructions from thecentral processor (or some intermediate device in communication with thecentral processor) and translating these signals into movement of thecutting armature 500. During the imaging process, the central processormay give instructions for moving the cutting armature 500 away from theplant stem 106 so that the position of the cutting armature does notoverlap or otherwise interfere with the image of the plant stem to betrimmed. The end of the cutting armature 500 may also includecapacitive, resistive, touch or pressure sensitive sensors for thepurpose of gathering additional data about the plant stem or trimmedareas, for example the water content of the plant stem.

The embodiments depicted in the attached FIGS. operate as follows.

As illustrated in FIGS. 2, 3 and 12, untrimmed plant stems 106 areloaded into chucks 108 in the transport mechanism 100. The process ofloading an untrimmed stem 106 into each chuck 108 may be accomplished byhand or using an automatic loading mechanism, which may include jaws orfingers for gripping an individual plant stem from a hopper or otherform of supply system, pushing it into the opening of an available chuckand then releasing the plant stem as the chuck grips it.

The cars 102, each including a rotatable chuck 108 and an untrimmedplant stem 106, are then moved by the conveyor system 104 from theloading/unloading area 116 and into the plant-trimming position 118.Upon arriving at the correct position, the conveyor system 104 is haltedand the rotation mechanism of the chuck 110 is engaged with a drivewheel driven by a motor 112 controlled by electrical signals orinstructions from the central processor 600 (or some intermediate devicein communication with the central processor).

As illustrated in FIG. 5, when the plant stem 106 reaches the correctplant-trimming position 118, the stem 106 is also supported in positionby the transparent platform 202 of the cutting support mechanism 200,which keeps the plant from drooping or bending out of the correctorientation while imaging and trimming are being done. This supportmechanism may be removed where, for example, the plant stem is orientedvertically, as shown in FIGS. 22 to 26, or on a stationary or movingconveyor.

As illustrated in FIG. 13, in the plant-trimming position 118, the plantstem 106 is positioned between the camera 300 lens and the imaging stage400. The camera 300 captures one or more images of the plant stem framedin front of the imaging stage 400. The plant stem 106 may also berotated while the camera 300 captures images from different angles inorder to provide the system with several choices for the bestorientation of the plant stem during trimming. Each of the imagescaptured from a different angle would be processed and evaluated usingone or several of the steps described below.

Image processing includes the following steps, which may be combined inany order or used separately to create a trim map—that is, a database offlower and leaf positions that instructs the cutting armature 500 whereto cut in order to trim away leaves while protecting the more valuableflowers. Basically, the image processing steps rely upon a series ofgeometric and other physical “markers” that are associated with theposition of flowers, leaves or stems and then assigns a probability toeach position based on the fit of these geometric markers. The entiremap of probability values, combined from each of the various imageprocessing steps, is then used to create the trim map. Edge detection,the Sobel-Feldman operator or filter, or other machine vision techniquesmay be utilized in this process. Additionally, machine learning andartificial intelligence techniques, such as, but not limited tosupervised learning, wherein the algorithm has been trained withnumerous sample images, may be used to produce increase confidence andaccuracy in the image processing algorithms. Supervised training ofthese networks would include, for example, identifying the location ofsugar and fan leaves, as well as flowers and branches on a cannabisplant. After entry of numerous images showing correctly identifiedfeatures, the computer is trained through software algorithm torecognize similar organic regions within a cannabis branch or plant, forexample. This improves upon the accuracy and confidence of previouslyand here on described algorithms.

As illustrated in FIGS. 14 and 15, the original image captured by thecamera 300 includes both the plant stem 106 and the reference markers404 on the imaging stage 400. The original image, however, is typicallytoo complicated and detailed for the image processing steps, and so theoriginal image may be simplified into a silhouette 700 by increasing theimage contrast and assigning a value for each pixel or group of pixelsas either “on” or “off,” which corresponds to either white or black.This silhouette is then further simplified using morphological imageprocessing to locate and either open or close small groups ofcontinguous pixels that do not match the value of a surrounding field.After this step, the original image 700 is reduced to a simplesilhouette or outline shape 702, which is the starting point for each ofthe leaf and flower recognition steps that follow.

As illustrated in FIGS. 16 and 17, one recognition step, called shapefitting, is an iterative, software-controlled computation whereby thecentral processor locates the largest continuous area of darkened pixelson the silhouette image 702 (not including the reference marks) and thencomputes and draws the largest circlular area 704 that will fit entirelyinside this area of darkened pixels. Specifically, by altering theintensity of an array of individually-addressable LEDs that form theimaging stage 400, the central processor 600 may be able to distinguishparts of the plant stem 106 that are dense or opaque to light, which aretypically the flowers. If an area of dense plant material is identifiedas having a high probability of being a flower, further analysis of itsshape is performed. Flowers typically form round, elongated geometriescharacterized by a relatively smooth surface. A “circle fit” imageprocessing algorithm is used to fit circles of independent diametersinto areas of contrast on the processed image. For a circle to fit, theentire shape must be enclosed—that is, no light or almost no light ispassing through the circular region. The result is an image of a flowerwith circles of varying diameters over-laid, as shown in FIG. 16. Thepositions and radii of the circles 704 are analyzed for their proximityto each other, their overall size, and their positions with regard tothe stem location. If a grouping of circles 704 forms a roughlyelliptical or oval region, is located in the region of the branch orstem, and is opaque or nearly opaque (that is, little porosity), it isgiven a score and identified as either one flower, multiple flowers, orother plant material.

As illustrated in FIG. 17, the central processor may also subtract ordisregard the area inside the circles, search the remaining shapes 706again to determine the next largest continuous area of darkened pixelsavailable on the image, and then compute and draw the largest circlethat will fit entirely inside this area. What remains after thecircle-fitting step is the silhouette image 702 marked with a series ofcircles 704 that correspond to most or all large areas of darkenedpixels on the silhouette image. These large areas of darkened pixels onthe silhouette image may correspond to the rounded or bulbous flowerareas of the untrimmed plant stem, and the circles fitted to these areasof darkened pixels may be used in the creation of a trim map to mark theboundaries where the cutting armature should be stopped to avoid damageto this part of the plant. Smaller areas of pixels 706 remaining afterthe deduction of the circular or oval areas may correspond to leaves,stems and other areas to be trimmed. To determine which of thesecircles, shapes, groups of shapes, and so forth, represents a singularflower, multiple flowers or other plant material, statistical analysisis completed to provide a probability score corresponding to each regionon the plant. This score may be included in the information collected inthe trim map, as described in greater detail below.

The shape-fitting step may be modified by using other shapes, includingellipses, ovals, triangles, other polygons or irregular blobs. Theshape-fitting step may also be modified by permitting a certain amountof overlap in the boundaries of the circles, thereby permitting circlesdrawn on the silhouette image to overlap each other or the boundaries ofthe dark and light pixels. Permitting a certain amount overlap mayenable the circles to be fitted somewhat “tighter” to the contours offlower shown in the silhouette image.

As illustrated in FIG. 20, another recognition step, called patternrecognition, is also an iterative, software-controlled computationwhereby the central processor 600 divides the entire boundary betweenlighted pixels and darkened pixels on the silhouette image 702 intodiscrete sections 708 and then compares the shape of each section'sboundary line to one or more reference shapes showing boundaries typicalof plant leaves. The central processor 600 also adjusts the rotation andscale of the reference shape or shapes to find the closest possible fitto the boundary section 708 being evaluated, with the closeness of thefit judged by the difference between the two boundary lines, as measuredin total area or number of pixels. When the central processor 600 findsa close fit between the boundary section 708 being evaluated and one ormore reference shapes, the boundary section 708 is marked for cutting onthe trim map. Further statistical analysis is completed at this step toconfirm that the identified shapes have a high probability of matchingthose of a leaf. The statistical analysis involves a comparison of thelength-to-width ratio of the flower shape as compared to the directionof the branch on which the flower shape is located. If the flower shapeis longer along the axis of the branch than it is wide, then it is morelikely to be an actual flower, whereas if the flower shape is shorteralong the axis of the branch than it is wide, then it is more likely tobe an actual leaf.

Another recognition step, called lighting modulation, incorporates theuse of the LED lighting of the imaging stage to provide an extremelybright backlit surface. First, an image with average light intensity istaken. Then, the LED lighting is brightened significantly, causingleaves and other thin plant material to become largely translucent tothe camera. Then the dense or thick materials, such as stems andflowers, remain visible to the camera. These two images are superimposedon each other, which removes the dense areas of flowers and stems seenin both images, while leaving only the features that appearedtranslucent to the camera in the second image. Because leaves allowpartial transmission of very bright light as compared to flowers, whichremain opaque, the process of comparing images using different lightintensities gives the central processor reliable information about whichportions of an image correspond to leaves as opposed to flowers. Thefeatures identified as leaves in this step may also be further analyzedwith one of the other image processing steps described herein.

As illustrated in FIG. 21, another image processing or recognition step,called leaf-tip recognition, is also an iterative, software-controlledcomputation whereby the central processor locates small areas ofdarkened pixels surrounded on more than one side by larger areas oflighted pixels and then computes and fits a small circle within eachdarkened area 710, each small circle identifying the tip of a leaf fortrimming. Leaf tips have a characteristic shape-small, thin areas thatnarrow to a sharp, triangular point—that can be located by the centralprocessor using simple search algorithms. The central processor thenmarks the position of each leaf tip 710 on the trim map for cutting.Again, statistical analysis and expected shape comparisons are used atthis step to confirm that the identified shapes have a high probabilityof matching the location of actual leaf tips.

As illustrated in FIG. 18, another image processing or recognition step,called branch identification, involves the indentification of branchesand stems through the use of edge detection techniques to recognizestraight lines of the plant stem images. Of course, very few branches orstems are straight in nature, and as such the process generally resultsin a number of discrete lines that must be knit together to describe afull branch section. To do this, a separate algorithm looks at thelocation of end points and start points of line sections. If, forexample, an end point is in close proximity to a start point, the angleof the computed edges are similar, and there are no detected flowers atthe intersection, these two lines are assumed to form a branch section.Several additional factors influence grading of individual edges,including the locations of leaves, smaller diameter stems or branchesand flower locations.

As illustrated in the flow-chart of FIG. 19, each of the imageprocessing steps described above—shape fitting, pattern recognition,lighting modulation, leaf-tip recognition and branch identification—maybe used alone or in combination with any of the other processing steps,or other steps, to create a trim map for guiding the cutting armature.Multiple image processing steps may be combined and applied to a singleimage to obtain a more accurate trim map, as illustrated in FIG. 21. Thetrim map includes information on the locations of areas of the plantstem to be removed through trimming and areas of the plant stem to beavoided during the cutting process. When the system utilizes more thanone image processing step, the data from each image processing step maybe combined using a probability function such that areas of the plantstem that are identified for cutting based on only one image processingmethod will be assigned a low “score” or “probability” for trimmingwhile areas of the plant stem identified based on two or three imageprocessing methods will be assigned a higher score or probability fortrimming.

The trim map is used by the central processor to construct a pattern formoving the cutting armature 500 to various locations along the plantstem 106 and thereby trim the stem's leaves efficiently andautomatically while avoiding any damage to the valuable flower andtrichomes. Following this cutting pattern, the central processor 600then moves the cutting armature 500 from the rest position at one sideof the imaging stage 400 into various locations along the plant stem106. The cutting armature's rotating blade 506 and vacuum system 508 areactivated during this process so that as the slit or opening 504 of thecutting armature 500 comes into contact with various parts of the plantstem 106, the leaves at that location will be sucked very briefly intothe opening 504 and trimmed off. As the cutting armature 500 moves closeto the plant stem, the cylindrical chamber 502 may come into contactwith and even push against the support platform 202. As statedpreviously, the support platform 202 is able to slide backwards inresponse to pressure from the cutting armature 500, and as soon as thecutting armature retreats, a bias spring or equivalent mechanism 204connected to the support platform 202 moves the platform back into itsoriginal, extended position. Thus the support platform 202 is able tokeep the plant stem 106 from bending or drooping throughout the imagingand trimming process without interfering with either function.

The trim map may also include instructions for rotating the slit 504 ofthe cutting armature 502, changing the angle between the long axis ofthe cutting armature and the axis of the plant stem 106, or changing theangle of scissors at the end of the cutting armature, which areordinarily substantially perpendicular. Rotating the slit 504 orchanging the angle of the cutting armature cylinder 502 or scissors inany axis may give the system additional capability for reaching“overhangs” or other difficult-to-trim areas of the plant stem 106without damage to the flower or trichomes. The trim map may also includeinstructions for various automatic movements to assist in the trimmingprocess. For example, the trim map may include instructions for thecutting armature 500 to move several times up and down (relative to thelong axis of the plant stem) by 2 mm to account for the fact that thetrim pattern disclosed through image processing may not correspondprecisely to the locations of actual leaves. This brief 2 mm up-and-downmovement does not risk any significant damage to the valuable parts ofthe plant stem and is effective in grabbing and trimming leaves thatremain just outside of the cutting armature's reach at its originallocation on the trim map. Other automatic movements to assist thetrimming process may also be added to the trim map by the centralprocessor, such as making a slight rotation (±5°) of the cutting slit orthe plant stem.

Once the central processor completes the trimming stage (that is, afterthe processor 600 moves the cutting armature 500 through all thelocations marked on the trim map for cutting), the central processormoves the cutting armature away from the plant stem and back to its“rest” position on the opposite side of the imaging stage 400. Thecentral processor 600 may then capture another image of the plant stem106 to compare to the image captured before the trimming process 700.This comparison serves to confirm that all the areas marked on the trimmap have been correctly trimmed, and if the comparison discloses thattrimming was not completed correctly, the central processor 600 may thenengage the cutting armature 500 for a second pass at the plant stem 106in the areas where trimming was incomplete.

Additionally, imaging may be accomplished continuously during thetrimming process, whereby the plant stem 106 is imaged multiple times,the image is processed as described above and the trim map is alteredwith new information to account for changes in the plant stem, leafposition and progress by the cutting armature throughout the trimmingprocess. Continuous and intermittent imaging may employ any number ofsensors including digital photography cameras with CMOS and CCD sensorsusing visible, infrared and other spectra. Capacitive, resistive, touchand pressure sensors may also be used to augment the imaging informationavailable to the central processor. Laser range finding, includingwithout limitation LIDAR sensing, may also be used in scanning anddifferentiating features on the plant stem. Thus, the data collected andanalyzed to determine and segment agricultural features and locationsmay not be limited to 2D data, but may also include 3D data gathered asa result of a structured light, laser range finding, or other method ofobtaining data in 3D dimensions. This data is then processed in the formof a point cloud dataset.

After determining that the trimming process has been successfullycompleted, the central processor 600 then rotates the chuck 108 holdingthe plant stem 106 to another orientation, exposing an area of the plantthat was not exposed or less exposed in the previous orientation. Theamount of rotation will depend on the number of rotations chosen foreach plant stem—that is, for one rotation cycle per plant, the plantwill be rotated through 180 degrees; for two rotation cycles, 120degrees; for three rotation cycles, 90 degrees, and so forth.Preferably, the central processor will make three to five of theserotation cycles for each plant stem.

Additionally, rotation of the plant stem 106 may be accomplishedcontinuously during the trimming process, whereby the plant stem isrotated multiple times, the image is captured and processed at eachpoint in the rotation as described above and the trim map is alteredwith new information corresponding to the differing views available asthe plant stem is rotated and trimmed. Continuous rotating combined withcontinuous or intermittent imaging may permit the creation of athree-dimensional image of the plant stem and its leaves. Imageprocessing would then proceed in the same manner as described above, butwith three-dimensional shapes (such as spheres) replacingtwo-dimensional shapes (such as circles) in the description above.

After rotating the chuck 108 to a new orientation, the process of imagecapture, image processing, trimming and verification begins again, asdescribed in detail above. After the last of the chuck rotations iscompleted and the successful trimming verified, the plant stem 106 isconsidered trimmed and the conveyor moves the trimmed plant stem 116 outof the way. Another untrimmed plant stem 106 is moved by the conveyorinto the correct trimming position, and the process begins again onanother stem.

The foregoing descriptions of specific embodiments of the presentinvention have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit theinvention to the precise forms disclosed, and obviously manymodifications and variations are possible in light of the aboveteaching. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical application,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications as aresuited to the particular use contemplated. It is intended that the scopeof the invention be defined by the Claims appended hereto and theirequivalents.

Moreover, it is to be appreciated that the terms “plant stem” and“plant” referred to herein are in the appended claims is to beinterpreted broadly to include any harvested portion of any plant thatmay be used for any commercial or agricultural purposes, and includeswithout limitation any fruits, nuts, flowers, bulbs, leaves, heads,seeds, pods, shoots, vegetables or any other useful part of any plant orany portion thereof.

I claim:
 1. An automated device for trimming an untrimmed plant steincomprising: a transport mechanism for holding and moving the plant stem;a camera and an imaging stage; a cutting armature having cuttingstructure for trimming the plant stem; and a central processor; whereinthe transport mechanism moves the plant stem proximate to the imagingstage for image capture; the camera disposed toward the imaging stage tocapture at least one image of the plant stem; the central processorreceiving the at least one image from the camera to create a trim mapusing at least some data gathered from the at least one image; and thecentral processor moving the cutting armature as guided by the trim mapto position the cutting structure against the plant stem for trimmingthe plant stem whereby the untrimmed plant stem is changed to a trimmedplant stem.
 2. The device of claim 1 wherein the transport mechanismincludes at least one car mounted to a conveyor system, the car having achuck to releasably hold the plant stem at the imaging stage.
 3. Thedevice of claim 2 wherein the chuck is rotatably mounted to hold theplant stem at different orientations, and the rotation of the chuckbeing driven by a motor receiving signals from the central processor. 4.The device of claim 3 wherein there are a series of cars on the conveyorsystem, and the chucks are mounted horizontally.
 5. The device of claim3 wherein there are a series of cars on the conveyor system, each carhaving a chuck and the chucks being vertically mounted, and the imagingstation including a vertically mounted backlighted surface.
 6. Thedevice of claim 1 wherein the imaging stage includes a backlightedsurface to illuminate the plant stem, and reference markers being on thebacklighted surface.
 7. The device of claim 6 including a supportplatform mounted on the backlighted surface, and the support platformbeing made of a transparent/translucent material to permit light to passthrough the platform.
 8. The device of claim 7 wherein the supportplatform is slidable toward and away from the cutting structure of thecutting armature, the support platform being spring biased toward anextended position and being movable by the cutting armature to aretracted position upon contact with and movement by the cuttingarmature, the support platform including a horizontal wall on which theplant stem would be placed, and a vertical fence extending upwardly fromthe horizontal wall to provide a stop for ensuring that the plant stemremains on the support platform.
 9. The device of claim 1 wherein thecutting armature includes a hollow chamber having an elongated generallyvertical slit in its outer wall to provide access to the interior of thechamber, the cutting structure being mounted in the hollow chamber, asource of reduced air pressure communicating with the interior of thehollow chamber to draw in plant parts from the plant stem and removingthe plant parts when the plant parts are cut from the plant stem, andthe hollow chamber having a lower end which slides along the imagingstage and an upper end receiving control signals from the centralprocessor.
 10. The device of claim 9 wherein the cutting structure is alaser cutter.
 11. The device of claim 9 wherein the cutting structurewithin the hollow chamber is a rotating blade, a stationary blade beingmounted adjacent to the slit, and the slit being axial.
 12. The deviceof claim 9 wherein the hollow chamber is rotatable, and the hollowchamber having a sensor for gathering data about the plant stem.
 13. Thedevice of claim 9 wherein the cutting armature is a six axis roboticarmature.
 14. The device of claim 9 wherein the transport mechanismincludes at least one car mounted to a conveyor system, the car having achuck to releasably hold the plant stem at the imaging stage, the chuckbeing rotatably mounted to hold the plant stem at differentorientations, the rotation of the chuck being driven by a motorreceiving signals from the central processor, the imaging stage includesa backlighted surface to illuminate the plant stem, and referencemarkers being on the backlighted surface.
 15. A method for the automatedtrimming of an untrimmed plant stem comprising transporting an untrimmedplant stem to an imaging stage, using a camera to capture at least oneimage of the plant stem, sending the at least one image to a centralprocessor which creates a trim map which constructs a pattern for movinga cutting armature along the plant stem, and trimming the plant stem bycutting structure of the cutting armature to change the untrimmed plantstem to a trimmed plant stem.
 16. The method of claim 15 wherein theplant stem is transported to the imaging stage by a conveying systemhaving at least one car with a rotatable chuck, and the plant stem beingremovably mounted in the chuck.
 17. The method of claim 16 wherein afterthe trimming has begun while the plant stem is in one orientation thechuck is rotated to a new orientation and more images are taken at thenew orientation.
 18. The method of claim 15 wherein the trim map isconstructed by a technique selected from the group consisting of shapefitting recognition, pattern recognition, lighting modulation, leaf tiprecognition, branch identification and supervised learning.
 19. Themethod of claim 15 wherein the plant stem is placed on the imagingstation with the imaging station being in a horizontal position andformed from a backlighted surface to illuminate the plant stem, theplant stem being mounted at the imaging station on a support platformmade from a transparent/translucent material which permits light to passthrough the platform, the platform being in an extended positiondisposed toward the cutting armature, and the support platform beingmovable to a retracted position by the cutting armature pressing againstthe support platform.
 20. The method of claim 15 wherein the imagingstage is a vertically disposed backlighted surface, and the plant stemis mounted in a vertical position.
 21. The method of claim 15 whereinthe plant stem is cannabis having a flower and plant parts, and thecannabis is trimmed of plant parts leaving the flower on the plant stem.